Papers with thresholding operator
Pruning Redundant Mappings in Transformer Models via Spectral-Normalized Identity Prior (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Spectral-normalized identity priors (SNIP) is a structured pruning approach for a Transformer model. |
| Approach: | They propose a structured pruning approach which penalizes an entire residual module toward an identity mapping. |
| Outcome: | The proposed method improves on 5 GLUE benchmark tasks while maintaining comparable performance. |